feat: add distributed expert workers - #551
Conversation
|
Thanks for splitting this out. Two blockers, and I want to be straight about both. (1) Priority: distributed expert workers are breadth (multi-machine), and the focus right now is depth on single-consumer-box models — so this isn't where attention is going this week. (2) Correctness, and this is the real one: |
The cluster registry subclasses openai_server.APIHandler, which enforces the DNS-rebinding Host guard (#SEC-7). That guard correctly rejects non-loopback Host headers, but a cross-host expert worker registers from its LAN IP, so the MVP (coordinator + remote workers) 403'd at discovery -- a regression the hardening introduced relative to the original JustVugg#551. Plumb the same escape hatch coli serve already exposes (JustVugg#597): accept --allowed-host (append, default from COLI_ALLOWED_HOSTS) on 'coli cluster coordinator' and thread it through cmd_cluster_coordinator -> cluster.serve -> ClusterServer.allowed_hosts. Default stays loopback + bind address; the guard is unchanged unless the operator opts a host in.
|
Closing in favour of #1036, which is this work carried forward rather than replaced. It keeps your Your #550 is where the token-exact discipline in this area came from, and #1036 mirrors it explicitly. Thanks for the original design — it's the one that shipped. |
The cluster registry subclasses openai_server.APIHandler, which enforces the DNS-rebinding Host guard (#SEC-7). That guard correctly rejects non-loopback Host headers, but a cross-host expert worker registers from its LAN IP, so the MVP (coordinator + remote workers) 403'd at discovery -- a regression the hardening introduced relative to the original JustVugg#551. Plumb the same escape hatch coli serve already exposes (JustVugg#597): accept --allowed-host (append, default from COLI_ALLOWED_HOSTS) on 'coli cluster coordinator' and thread it through cmd_cluster_coordinator -> cluster.serve -> ClusterServer.allowed_hosts. Default stays loopback + bind address; the guard is unchanged unless the operator opts a host in.
The cluster registry subclasses openai_server.APIHandler, which enforces the DNS-rebinding Host guard (#SEC-7). That guard correctly rejects non-loopback Host headers, but a cross-host expert worker registers from its LAN IP, so the MVP (coordinator + remote workers) 403'd at discovery -- a regression the hardening introduced relative to the original JustVugg#551. Plumb the same escape hatch coli serve already exposes (JustVugg#597): accept --allowed-host (append, default from COLI_ALLOWED_HOSTS) on 'coli cluster coordinator' and thread it through cmd_cluster_coordinator -> cluster.serve -> ClusterServer.allowed_hosts. Default stays loopback + bind address; the guard is unchanged unless the operator opts a host in.
2 participants
Summary
coli cluster coordinatorcoli cluster workerand coordinator discovery forcoli serveCorrectness and scope
The single-machine path remains unchanged unless
CLUSTER_WORKERSor--cluster-coordinatoris configured. The control-plane tests cover registration, heartbeat, stale topology filtering, and expert discovery. Native C tests andmake colibripass on macOS.This PR is intentionally the expert-worker slice only. Dense activation sharding is in #550; WebGPU/browser workers will be proposed separately. The original bundled PR #380 remains open as requested and is not this PR's merge unit.